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SECTION  I 


INTRODUCTION 

1.  LIMITATIONS  OF  CLASSICAL  TECHNIQUES 

Pattern  recognition  Is  often  performed  In  autonomous  hardware  by  comparing 
Imagery  from  an  on-board  sensor  with  an  Ideal  or  reference  Image  stored  In  a 
memory.  Military  systems  frequently  are  required  to  recognize  patterns  even 
though  they  are  slightly  different  from  the  reference  for  which  the  system  was 
designed.  Signal  processing  techniques  to  overcome  certain  types  of  distor¬ 
tions  In  sensed  Imagery  have  been  described  In  the  literature.  Reference  1. 

The  most  common  distortions  are  scale  and  orientation  changes.  Classical 
matched  filters  are  unreliable  unless  the  reference  and  sensed  objects  are  very 
nearly  the  same.  When  the  peak  signal  to  RMS  noise  ratio  (SNR)  of  the  correla¬ 
tion  function  Is  used  as  a  figure  of  merit  for  the  holographic  matched  filter 
technique,  losses  In  SNR  of  27  db  can  occur  with  as  little  as  2  percent  scale 
difference  and  3.5  degrees  of  orientation  difference.  Reference  2. 

2.  THE  MELLIN-FOURIER  CORRELATION  PROCESS 

Distortions  such  as  scale  changes  and  rotations  of  the  pattern  of  the 
sensed  Image  may  be  overcome  by  using  the  proper  choice  of  geometrical  trans¬ 
formations.  Since  most  scale  Invariant  transformations  are  not  invariant  to 
translation,  the  Fourier  transform  magnitude  is  sometimes  used  as  the  first 
step  In  a  pattern  matching  scheme  to  achieve  shift  invariance  before  the  scale 
Invariant  geometrical  transformation  Is  performed.  The  scale  invariant  trans¬ 
formation  Is  achieved  In  practice  by  resampling  the  original  image.  This 
resampling  process  causes  spatial  distortion,  e.g.  warping,  and  is  therefore 
sometimes  called  space  variant.  Figure  1  shows  the  process  used  to  achieve 
Mel  1 1n-Fourier  (scale  and  shift  Invariant)  correlation  between  a  reference 
Image  and  a  sensed  Image.  The  first  step  Is  to  calculate  the  magnitude  of  the 
Fourier  transform  of  the  reference  and  sensed  images.  The  Mel  1  in  transform  Is 
not  Invariant  to  object  translation,  so  pattern  recognition  using  Mellin  trans¬ 
forms  Is  usually  done  using  the  magnitude  of  the  Fourier  transform  of  an 
object.  This  technique  uses  the  shift  invariance  of  the  Fourier  transform  to 
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Figure  1.  The  Mel! 1 n-Fourler  correlation  process 


provide  a  centered  Image  for  later  exponential  resampling.  The  magnitude  of 
the  Fourier  transform  Is  equivalent  to  the  diffraction  pattern  obtained  when  an 
Image  Is  used  as  the  aperture  of  an  optical  system.  This  pattern  contains  no 
phase,  l.e.  position.  Information,  and  as  a  result  Is  invariant  to  the  position 
of  the  pattern  within  the  system  field  of  view. 

The  next  step  in  the  Mel  1  In-Fourier  correlation  process  is  to  perform  the 
geometrical  transformation  necessary  to  changes  in  the  input  image.  The  geome¬ 
tric  transformation  used  in  this  research  maps  the  Fourier  transform  polar 
plane  into  a  rectangular  plane.  The  orthogonal  axes  in  the  new  rectangular 
coordinate  system  are  defined  as  the  logarithm  of  radial  distance  and  the  polar 
angle  in  the  original  Fourier  transform  plane.  These  axes  are  denoted  as  In 
(R)  and  0.  respectively.  The  quantity  R  is  the  radial  distance  from  the  zero 
frequency,  or  d.c.  term.  The  natural  logarithm  of  R  is  used  in  the  transforma¬ 
tion  because  a  multiplicative  scale  change  occurring  in  the  Fourier  transform 
plane  causes  a  linear  shift  in  the  In  (R)  axis.  The  quantity  0  is  the  angle 
measured  counterclockwise  from  the  horizontal  axis.  When  a  Fourier  transform 
image  is  mapped  using  this  geometric  transformation,  a  new  image  plane  is 
created  with  each  location  in  the  In  (R),0  plane  corresponding  to  one  or  more 
locations  in  the  Fourier  transform  plane.  Figure  2  shows  what  happens  in  the 
ln(R),0  plane  when  a  scale  change  occurs  in  the  original  image.  The  top  image 

in  Figure  2  is  the  ln(R),0  image  generated  by  a  10  x  10  reference  square.  The 

* 

ln(R),0  images  below  it  are  sensed  images  with  various  scale  changes  present. 
As  the  scale  of  the  sensed  image  changes,  the  pattern  in  the  ln(R),0  image 
simply  translates.  Features  in  the  ln(R),0  image  do  not  change  size  as  they 
would  in  the  Fourier  transform  image.  It  is  this  scale-to-translation  property 
of  the  1  n(R) ,0  mapping  that  is  exploited  in  the  scale  invariant  Mel  1  in-Fourier 
correlator. 

The  final  step  in  Mel  1  in-Fourier  correlation  is  a  standard  cross  correla¬ 
tion  calculation  performed  with  the  ln(R),0  reference  and  sensed  images  as 
inputs.  The  correlator  in  this  work  is  implemented  with  a  computer  simulation, 
and  cross  correlation  is  performed  using  transform  techniques  rather  than  by 
direct  calculation.  This  technique  requires  fewer  calculations  than  cross 
correlation  performed  directly.  The  output  of  the  Mell in-Fourier  correlator  is 
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Figure  2.  Effect  of  scale  changes  on  the  ln(R),9  Image 


a  cross  correlation  image  with  a  peak  value  whose  X  coordinate  reveals  the 
amount  of  scale  change  between  the  reference  and  sensed  images.  The  Y  coordi¬ 
nate  of  the  cross  correlation  peak  gives  the  amount  of  rotation  difference 
between  the  reference  and  sensed  images.  This  work  is  concerned  with  consider¬ 
ations  in  sampling  the  ln(R),B  image  and  the  scale  invariance  expected  of  a 
system  using  the  sequence  of  operations  described  in  Figure  1.  In  addition, 
the  effects  of  noise  added  to  the  reference  and  sensed  images  on  the  shape  and 
amplitude  of  the  cross  correlation  image  are  determined. 

3.  RESULTS  OBTAINED  BY  OTHER  WORKERS 

Cassasent  and  Psaltis,  Reference  3,  have  shown  how  space  variant  processors 
can  be  used  to  perform  scale  invariant  correlation.  With  their  technique, 
scale  and  rotation  changes  in  a  sensed  pattern  were  converted  into  a  transla¬ 
tion  by  use  of  the  logarithmic  coordinate  transformation  described  above. 
Standard  shift  invariant  correlation  methods,  i.e.,  Fourier  transform  methods, 
were  then  used  for  pattern  recognition  of  the  scaled  function. 

An  analysis  performed  by  Cassasent  and  Psaltis  showed  that  the  spatial 
distortion  introduced  by  the  logarithmic  transformation  resulted  in  increased 
space  bandwidth  requirements.  The  logarithmic  coordinate  transformation  is 
equivalent  to  resampling  an  image  at  exponentially  spaced  intervals,  and  the 
increase  in  space  bandwidth  (number  of  samples)  is  caused  by  the  oversampling 
which  takes  place  at  small  values  of  the  Input  coordinate.  Their  analysis 
considered  the  Increased  sampling  requirements  imposed  by  a  function  which 
moved  partially  out  of  the  sampled  domain  when  scaled  by  a  factor  A.  This  loss 
of  information  gave  rise  to  inaccuracies.  Space  bandwidth  requirements  were 
said  to  Increase  by  a  factor  of  5.3  when  100  percent  scale  change  (A=2)  was 
accommodated.  Reference  3. 

Analysis  done  by  Anderson  and  Call  ary.  Reference  4,  involved  a  function 
which  remained  entirely  within  the  sampled  domain  when  scaled  by  a  factor  A. 

In  this  case,  there  were  no  Inaccuracies  introduced  because  the  function  was 
defined  to  be  zero  outside  the  domain  of  interest,  and  no  information  about  the 
function  was  lost  when  scale  changes  occurred.  They  showed  that  the  increase 
in  space  bandwidth  could  be  minimized  by  careful  choice  of  the  constants  used 
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NORMALIZED  AMPLITUDE 


FOURIER  TRANSFORM  MAGNITUDE 


SENSED  20  X  20  SQUARE 


SPATIAL  FREQUENCY  (CYCLES/FRAME) 


Figure  5.  Amplitude  along  a  line  passing  through 
the  center  of  the  Fourier  transform  of  a  20  X  20 
square.  Normalized  from  0  to  255. 


19 


NORMALIZED  AMPLITUDE 


'  If*  V1  V 


*■'  W  l\  J 


FOURIER  TRANSFORM  MAGNITUDE 

260 
240 
220 
200 
180 
160 
140 
120 
lOO 
80 
60 
40 
20 
O 

O  20  40  60  80  lOO  120 

SPATIAL  FREQUENCY  (CYCLES/FRAME) 

Figure  4,  Amplitude  along  a  line  passing  through 
the  center  of  the  Fourier  transform  of  a  10  X  10 
square.  Normalized  from  0  to  255. 
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SECTION  III 

APPLYING  OPTIMUM  SAMPLING  TECHNIQUES 


1.  EXAMPLES  OF  OPTIMUM  SAMPLING 

The  Fourier  transform  patterns  In  Figures  4  and  5  have  been  generated  to 
Illustrate  the  technique  of  designing  a  transformation  which  maps  the 
Fourier  transform  space  Into  another  space  with  coordinates  R  and  0.  The 
patterns  In  Figures  4  and  5  represent  the  magnitude  of  the  Fourier  transform 
of  sqaures  of  sizes  10  by  10  and  20  by  20,  respectively.  The  Fourier 
transform  image  is  represented  by  a  plot  of  intensity  along  the  center  of 
the  image.  Figures  6  and  7  are  from  the  R,  0  mapped  images.  Study  of  the 
size  of  the  features  in  Figures  6  and  7  shows  that  these  features  change 
size  and  position  with  the  object  scale  change,  just  as  they  do  In  the 
Fourier  transform  Image  before  mapping.  In  this  case  the  scale  change 
encountered  is  a  factor  of  2.  Figures  80  and  9  are  from  the  In  (R),0  mapped 
images.  Here  the  features  present  in  Figure  8  simply  translate  to  a  new 
position  in  Figure  9;  they  do  not  change  size.  However,  new  information  is 
Introduced  from  the  right  in  Figure  9  which  was  no  present  in  Figure  8. 
Dlstorlton  is  also  introduced  in  both  figures  because  of  the  many-to-one 
mapping  that  occurs  at  small  values  of  radial  distance  in  the  Fourier  plane. 

In  producing  both  Figures  8  and  9,  the  same  transformation  parameters 
were  used;1.e.  the  mapping  was  goverened  by  the  same  set  of  equations.  For 
each  figure,  pixel  values  in  the  in  (R),8  space  were  derived  from  the  same 
calculated  position  In  the  Fourier  transform  space.  With  a  fixed  set  of 
transformation  equations,  a  scale  change  causes  new  information  to  enter  the 
transformed  image,  causing  a  decrease  in  the  cross  correlation  peak  value. 
This  difficulty  may  be  overcome  by  adapting  the  transformation  equations  to 
map  a  different  region  of  the  Fourier  transform  plane  into  the  new  space. 

In  Figures  8  and  9,  range  Information  was  used  to  choose  the  size  and 
position  of  the  annulus  in  the  Fourier  transform  plane,  and  the  annulus 
tracked  the  region  of  critical  object  information  over  a  wider  range  of 
scale  change.  Of  course,  spatial  resolution  in  the  transformed  image  will 
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Fourier  plane  rather  than  sampling  bandwidth  In  the  ln(R),0  plane.  It  may  be 
argued  that  knowledge  of  range  for  the  sensed  Image  would  allow  the  sensed 
Fourier  transform  to  be  scaled  appropriately,  thus  making  scale  Invariant 
schemes  unnecessary.  In  a  practical  system,  however,  range  information  is 
usually  only  approximate,  making  the  sensed  frequency  annulus  poorly  defined. 
Range  adaptation  Is  valuable  because  it  makes  a  correlator  more  tolerant  to 
errors  in  size  estimates  for  the  sensed  image.  Evidence  of  this  tolerance  is 
shown  in  the  examples  of  the  next  Section. 
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transform  plane.  For  a  sensor  with  a  field  of  view  of  3  degrees,  the  cutoff 
frequencies  for  a  6  meter  object  with  .5  meter  features  at  a  range  of  500 
meters  are 

upper  frequency  ■  (R  tan(F))/S 

-  (500/. 5)  tan  (3) 

*  52  cycles/ frame 

lower  frequency  *  (500/6)  tan  (3) 

a  4  cycles/ frame 

The  full  space  bandwidth  of  the  Mel  1  in-Fourier  correlation  resampling  system 
may  then  be  applied  to  an  annulus  between  4  and  52  pixels  from  the  center  or 
d.c.  term  in  the  Fourier  transform  plane.  The  transformation  equation  neces¬ 
sary  for  optimal  resampling  may  be  written  in  terms  of  radial  distances  in  the 
Fourier  transform  plane: 

R,-1n<R>-  ,n<W 

where 

Vilt  <  R  <  ^niax 

This  equation  is  used  to  generate  sample  coordinates  in  the  1  n(R) ,0 -  plane 
where  the  value  of  R(nl-n  is  the  lower  spatial  frequency  limit  derived  for  the 
specific  pattern,  4  pixels,  and  the  value  of  R  extends  to  the  upper  frequency 
limit,  Rmax,  of  52  pixels.  The  region  in  the  Fourier  plane  that  is  oversampled 
to  construct  the  logarithmic  data  record  does  not  begin  until  a  radius  of  4 
pixels  has  been  reached  in  the  Fourier  plane,  thereby  minimizing  the  over- 
sampling  that  occurs  at  small  values  of  the  Fourier  radial  coordinate.  Since  no 
data  of  Interest  lie  beyond  a  radius  of  52  pixels  in  the  Fourier  plane,  the 
sampling  that  occurs  at  52  pixels  from  the  center  can  be  made  to  just  meet  the 
Nyquist  rate. 

The  analysis  above  has  shown  how  system  space  bandwidth  may  be  conserved  by 
using  range  adaptation  to  define  the  proper  annulus  in  the  Fourier  plane  and 
then  adapting  the  transformation  equations  to  map  the  annulus  into  the  ln(R),0 
plane.  When  range  adaptation  is  performed  as  a  normal  part  of  signal  process¬ 
ing,  like  proper  windowing,  the  space  bandwidth  requirements  of  a  pattern 
recognition  system  may  be  expressed  in  terms  of  spectral  resolution  in  the 


relevant  Information  remains  within  a  fixed  domain  when  a  scale  change  occurs, 
and  only  translates,  the  lower  curve  shows  that  sampling  requirements  in  X 
space  are  less  severe,  i.e.,  samples  are  further  apart,  than  if  the  transforma¬ 
tion  equations  are  left  fixed  and  the  upper  curve  is  used  to  generate  sample 
coordinates  in  X  space.  The  number  of  samples  in  X*  space  necessary  to  meet 
the  Nyquist  conditions  in  X  and  still  keep  oversampling  to  a  minimum  at  Xml-n 
determines  the  space  bandwidth  requirements  of  the  scale  invariant  correlation 
system.  The  theory  developed  by  Anderson  and  Callary  makes  use  of  the  assump¬ 
tion  that  no  information  Is  allowed  to  leave  the  interval  between  X„.  and  X 

min  max 

when  a  scale  change  occurs.  Reference  9.  This  means  that  there  are  no  accuracy 
problems  such  as  the  ones  in  the  analysis  of  Cassasent  and  Psaltis,  Reference 
10.  This  is  the  basis  for  the  savings  in  space  bandwidth  claimed  in  Reference 
9. 

b.  Sample  Adaptive  Cal ulation 

An  example  will  show  how  range  adaptation  may  be  applied  to  a  specific 
pattern  to  derive  a  spatial  frequency  filter  for  use  in  a  Mel  1  in-Fourier  corre¬ 
lation  scheme.  The  filter  is  a  window  applied  to  the  Fourier  transform  plane, 
defining  the  area  to  be  resampled.  The  width  and  boundary  values  of  the 
region  are  determined  by  the  nature  of  the  object  to  be  recognized  and  the 
expected  range  at  which  It  will  be  encountered.  In  the  Fourier  plane,  the 
filter  specification  will  be  two  radial  values  determined  from  cal culati ions 
of  expected  object  size  in  the  sensor  field  of  view.  The  resampling  scheme 
will  then  be  applied  to  the  region  to  minimize  oversampling  in  the  Mel  1  in- 
Fourier  correlation.  For  this  example,  it  Is  assumed  that  other  sensors  have 
provided  approximate  range  as  R  meters,  and  that  the  image  and  Fourier 
transform  are  made  up  of  N  x  N  arrays.  If  the  sensor  field  of  view  is  F 
degrees,  then  the  number  of  pixels,  P,  occupied  by  a  feature  of  size  S  is 

P  *  (NS)  /  (R  tan  (F)) 

In  a  reference  Image,  a  square  feature  of  size  P  produces  a  first  spectral  lobe 
which  goes  to  zero  at  N/P  pixels  from  the  center  pixel  in  the  Fourier  plane. 

The  first  zero  crossing  defines  a  boundary  value  for  the  region  chosen  for 
,'esampllng.  Two  boundaries  of  the  spatial  frequency  bandpass  may  be  estab¬ 
lished  from  estimated  object  feature  sizes  in  the  image  plane.  The  two  cutoff 
frequencies  will  then  define  an  annulus  of  spatial  bandpass  in  the  Fourier 
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The  sampling  In  the  Fourier  transform  plane  extends  over  an  Interval  between 
Xj^  and  and  the  samples  have  a  non-uniform  spacing  along  the  X  axis. 

The  widest  spacing  In  X  occurs  at  Xmax  and  the  closest  spacing  occurs  at  X^. 
Since  the  sample  spacing  In  X  Is  most  dense  at  X^,  It  Is  best  to  assure  that 
this  part  of  the  sampling  curve  has  samples  In  X  that  are  as  far  apart  as 
possible  If  the  system  bandwidth  Is  to  be  most  effectively  used.  At  the  same 
time,  the  Interval  at  X^  must  be  no  greater  than  the  Nyqulst  distance  In 
order  to  preserve  all  the  Information  present  In  the  original  Image.  The 
slope  of  the  logarithmic  curve  at  Xmax  Is  determined  by  the  Nyqulst  require¬ 
ment,  and  the  space  bandwidth  of  X1  space  Is  largely  determined  by  the  slope 
of  the  sampling  curve  at  X^. 

When  Mel  1  In-Fourier  correlation  Is  done  In  two  dimensions,  the  X*  axis  In 
Figure  3  represents  the  ln(R)  axis  In  the  ln(R),8  plane.  The  X  axis  repre¬ 
sents  the  radial  frequency  coordinate,  or  R,  In  the  Fourier  transform  plane. 

If  the  chosen  region  In  the  Fourier  transform  plane  does  not  change  position 
and  adapt  to  the  new  object  spatial  frequencies  as  the  object  scale  changes. 
Information  may  enter  or  leave  the  annulus  defined  by  earlier  size  estimates. 
The  new  Information  appearing  In  the  In  (R),  0  image  will  cause  the  cross 
correlation  with  a  reference  pattern  to  be  reduced.  To  prevent  the  reduction 
of  the  cross  correlation  peak,  the  chosen  region  in  the  Fourier  transform  plane 
must  adapt  by  changing  position  when  a  scale  change  occurs  In  the  object. 
Specifically,  system  space  bandwidth  may  be  conserved  by  using  range  adaptation 
In  two  ways: 

-  Using  expected  object  range  to  select  the  annulus  of  spatial  frequencies 
In  the  Fourier  transform  plane  over  which  resampling  will  be  done,  and 

-  Using  the  analysis  developed  for  minimizing  the  slope  of  the  resampling 
curve  at  X^  to  modify  the  transformation  equations  to  map  the  spatial 
frequency  annulus  Into  the  full  space  bandwidth  of  the  ln(R),0  plane. 

These  steps  will  assure  Nyqulst  sampling  over  the  chosen  region  in  the  Fourier 
transform  plane,  and  will  minimize  oversampling  at  the  smallest  value  of  the 
Input  coordinate. 

a.  Adapting  The  Coordinate  Transformation 

The  upper  curve  In  Figure  3  represents  sampling  done  using  a  fixed  set 
of  transformation  equations  In  which  x^  *  .10.  The  lower  curve  represents 
sampling  done  with  an  adaptive  tranformatlon  equation  with  X^  *  .50.  If  the 


SPACE  SAMPLE  POINT 


SAMPLING  IN  X  AND  X'  SPACE 


DATA  COMPRESSION  AND  DILATION 


X  SPACE  SAMPLE  POINT 
□  LN(X/.1)  +  LN(X/.5) 

Figure  3.  Sample  spacing  in  the  Fourier  transform 
plane,  X  space,  and  in  the  ln(R),9  plane,  X*  space. 
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perfect.  In  cases  where  the  spatial  frequency  filter  In  the  correlation  plane 
Includes  the  center,  or  d.c.,  point,  the  correlation  coefficient  should  be 
normalized  to  the  energy  In  the  reference  Image  using  the  expression 

C(X,Y)  -  f  [SfX’.r)  RtX+X'J+Y'ndX'dY1 _ 

[J  [S(X',Y')lfdX'dY' ..  {[R{X\Y,)]zd'X'dY,]]1/2 
where 

C(X,Y)  3  correlation  Image  plane 
R(X,Y)  *  reference  Image 
S(X,Y)  3  sensed  Image 

X',Y'  3  coordinates  used  as  dummy  variables  In 
computing  the  cross  correlation 

This  expression  was  used  for  calculation  of  the  correlation  coefficient 
throughout  this  program. 

2.  THEORY  FOR  EXTENDING  THE  NORMAL  SCALE  INVARIANCE 

Use  of  the  MeTUn  transform  for  pattern  recognition  would  seem  to  require 
more  space  bandwidth,  1.e.,  more  samples,  than  required  to  obtain  the  original 
reference  pattern.  A  brief  analysis  of  the  resampling  process  will  show  how 
the  requirements  for  Increased  space  bandwidth  may  be  minimized. 

The  theory  developed  to  make  full  use  of  sampling  capability  Is  best  Illus¬ 
trated  by  an  example  in  one  dimension.  Figure  3  shows  the  relationship  between 
sample  spacing  In  the  Fourier  transform  plane,  designated  as  X  space,  and 
sample  spacing  In  the  in  (R),8  plane,  designated 

as  X'  space.  The  samples  In  X'  space  occur  at  equally  spaced  Intervals  of  the 
variable  X*.  The  coordinate  transformation  that  achieves  scale  Invariance  is 
represented  by  a  curve  that  follows  the  functional  form 

*V  '"<V  w 

where 

X.  3  sample  In  X  space 

k  th 

X'k*  k  sample  In  X’  space 

3  minimum  value  of  spatial 

frequencies  In  Fourier  plane 


functions.  In  one  dimension,  the  cross  correlation  of  two  records  of  length  N 
results  In  2N  valid  data  points.  If  transform  techniques  are  used  to  perform 
the  cross  correlation,  each  Fourier  transform  computed  must  be  truncated  to 
one-half  Its  original  size  before  multiplication  and  Inverse  transforming. 
Otherwise,  an  effect  called  cyclic  correlation  causes  errors  In  the  cross 
correlation  plane.  Reference  7.  This  effect  Is  similar  to  aliasing,  but  occurs 
In  the  cross  correlation  domain  Instead  of  the  spatial  frequency  domain. 

c.  Considerations  for  Geometric  Transformations 

The  spatial  frequency  characteristics  of  the  Fourier  transform  magni¬ 
tude  of  the  Input  pattern  must  be  considered  If  system  space  bandwidth  Is  to  be 
used  most  effectively.  The  symmetry  of  the  Fourier  transform  magnitude  may  be 
used  to  reduce  space  bandwidth  requirements  by  taking  all  samples  from  the 
upper  half  plane  of  the  diffraction  pattern.  The  upper  half  plane  of  the 
diffraction  pattern  contains  all  of  the  recoverable  shape  Information  about  an 
object,  even  though  position  Information  Is  lost.  All  available  space  band¬ 
width  (sampling  capability)  should  be  devoted  to  this  region  when  the  geometric 
transformation  Is  done.  Only  a  portion  of  the  Fourier  transform  plane  carries 
relevant  shape  Information,  and  a  geometric  transformation  that  adapts  to  that 
frequency  band  and  devotes  full  resolution  to  the  most  relevant  spatial  fre¬ 
quencies  Is  most  likely  to  provide  the  discrimination  desired.  In  this  way, 
invariance  to  scale  may  be  extended  to  cover  more  than  the  range  obtained  with 
a  transformation  which  maps  over  a  fixed  frequency  band. 

Previous  Investigators  have  designed  the  polar  transformation  to  oper¬ 
ate  over  a  fixed  spatial  bandpass  In  the  Fourier  transform  plane,  thereby 
limiting  the  Inherent  scale  Invariance  of  the  geometrical  transform.  Reference 
8.  Since  size  Information  about  the  expected  object  Is  often  available,  it  may 
be  used  to  select  the  most  appropriate  area  In  the  Fourier  transform  plane  and 
thereby  extend  the  normal  scale  Invariance  of  the  Mel  1  in-Fourier  correlator. 

d.  Choosing  a  Figure  of  Merit 

In  evaluating  a  Mel  1  In-Fourier  correlator,  a  figure  of  merit  must  be 
used  which  reflects  the  pattern  recognition  ability  of  the  system.  Experimen¬ 
tal  considerations  such  as  the  tendency  of  many  sensors  to  drift  or  change  the 
average  value  of  an  Image  over  time  may  cause  the  cross  correlation  peak  value 
to  decrease  even  though  the  pattern  match  with  a  reference  image  is  nearly 


further  use  for  pattern  recognition  may  produce  misleading  results.  Alterna¬ 
tively,  the  number  of  data  points,  N,  may  be  so  small  that  the  widely  spaced 
samples  do  not  completely  describe  the  shape  of  the  spectral  curve,  even  though 
the  Nyqulst  condition  has  been  met  and  the  samples  are  numerically  accurate. 

Thus,  two  considerations  apply  to  all  Invariant  pattern  recognition  schemes 
which  use  digital  transform  techniques: 

1.  Spectral  resolution  must  be  sufficient  to  show  identifying 
features  of  the  pattern. 

2.  Nyqulst  conditions  must  be  met  to  assure  numerical  accuracy 
of  spectral  samples. 

b.  Windowing  and  Truncation 

Practical  application  of  the  principles  above  involves  data  windowing 
and  truncation.  These  two  effects  are  manifest  In  the  spatial  domain  and 
spatial  frequency  domain,  respectively.  Windowing  of  Input  data  must  be  per¬ 
formed  to  minimize  the  effects  on  the  Fourier  transform  of  a  finite  input 
record  length.  The  DFT  produces  an  output  that  Is  always  a  combination  of  the 
actual  data  spectrum  and  the  window  spectrum.  Reference  6.  The  window  Is 
usually  a  rectangular  function  of  unity  amplitude  which  Is  multiplied  by  the 
data.  The  DFT  results  In  the  convolution  of  the  spectrum  of  the  window  with  the 
spectrum  from  the  pattern  of  Interest.  The  DFT  Is  periodic,  Nyqulst  conditions 
require  that  the  highest  spatial  frequency  present  In  a  record  of  N  samples 
be  N/2  cycles  per  frame.  The  abrupt  transitions  at  the  edges  of  a  rectangular 
window  cause  very  high  spatial  frequencies  to  occur  In  the  spectrum.  These 
spectral  components  can  be  significant  at  N/2  cycles  and  can  cause  aliasing 
errors  to  occur.  One  way  to  reduce  the  components  at  N/2  cycles  Is  to  make  the 
edge  transitions  less  abrupt  by  using  one  of  the  well  known  smoothing  windows 
such  as  the  Hanmlng,  Hanning,  or  cosine  functions.  The  need  to  condition  all 
data  with  a  window  before  computing  the  DFT  stems  directly  from  the  Nyqulst 
sampling  condition.  Proper  windowing  assures  numerical  accuracy  of  all  Fourier 
transform  samples  computed  using  the  DFT,  Reference  6.  The  window  used  In  this 
research  tapered  all  Images  to  1/2  the  original  value  at  each  edge,  reducing 
the  frequency  components  at  the  edge  of  the  Fourier  transform  plane. 

In  a  way  completely  analogous  to  windowing,  truncation  of  Fourier 
transform  records  avoids  errors  In  computing  the  cross  correlation  of  two 


SECTION  II 
BACKGROUND 

During  this  Investigation,  It  was  recognized  that  use  of  the  Mel  1  In-Fourier 
correlation  principle  would  require  careful  application  of  the  Discrete  Fourier 
Transform  (DFT).  The  paragraphs  that  follow  discuss  several  aspects  of  the  DFT 
and  show  how  to  properly  apply  it  to  evaluate  a  pattern  recognition  system. 

1.  EXPERIMENTAL  TECHNIQUES 

a.  Methods  Used  to  Calculate  Accurate  DFTs 

As  mentioned  earlier.  In  a  pattern  recognition  system  which  uses  trans¬ 
form  techniques  to  achieve  Invariance  to  translation,  scale,  and  rotation,  the 
Fourier  transform  Is  usually  used  Initially  to  provide  translation  invariance. 
Actual  pattern  recognition  is  performed  on  the  magnitude  of  the  Fourier  trans¬ 
form  of  an  object  rather  than  the  object  itself.  When  the  Fourier  transform  Is 
computed,  the  spectral  resolution  achieved  may  be  insufficient  for  reliable 
pattern  recognition  even  though  the  Nyqulst  criteria  have  been  met.  The  two 
Issues  of  spectral  resolution  and  Nyqulst  sampling  must  be  considered  separ¬ 
ately. 

In  a  DFT  algorithm,  the  Input  record  consists  of  N  data  points,  each 
point  representing  T  units,  producing  a  record  length  NT  units  long.  The 
output  of  the  DFT  algorithm  will  also  contain  N  data  points,  each  point  repre¬ 
senting  a  spatial  frequency  Interval 

delta  f  *  l/NT 

The  discrete  frequency  spectrum  is  repeated  periodically  at  integer  multiples 
of  the  sampling  frequency  fs  »  1/T,  with  the  magnitude  of  the  spectrum  being 
symmetric  about  the  folding  frequency 

f  «  (N/2)  delta  f  -  1/2T 

This  folding  frequency  is  a  direct  result  of  sampling  the  input  signal,  and 
gives  rise  to  the  Nyqulst  requirement  of  sampling  at  twice  the  highest  spatial 
frequency  present  In  the  Input  signal.  If  sampling  is  done  at  less  than  the 
Nyqulst  rate,  an  effect  called  aliasing  causes  Inaccuracies  in  the  computed 
spectrum. 

The  DFT  of  an  Input  record  may  be  calculated  with  a  resolution  of  delta  f 
even  though  the  Nyqulst  condition  has  not  been  met.  However,  the  values  com¬ 
puted  for  the  spectrum  will  not  be  accurate  because  of  aliasing,  and  any 


scaling  all  Imagery  amplitudes  between  0  and  255  each  time  a  transformation  Is 
performed.  The  response  to  high  noise  levels  Is  used  to  evaluate  the  Immunity 
to  background  clutter.  Trials  conducted  with  real  sensed  imagery  provide 
a  realistic  test  for  a  scale  Invariant  pattern  recognition  system  Intended  for 
tactical  applications.  These  trials  show  that  the  scale  and  rotation  invari¬ 
ance  predicted  by  theory  can  be  appl  yl  In  practice.  The  space  bandwidth  of 
the  correlator  is  varied,  and  the  effects  on  the  correlation  images  suggest 
design  approaches  for  equipment  using  Mell in-Fourier  principles. 

Experimental  techniques  for  both  zooming  and  downsampling  the  reference 
Image  are  developed  to  be  able  to  provide  a  sensed  image  of  exactly  the  proper 
scale  and  orientation  for  Mel  1  In-Fourier  correlation.  In  this  way,  the  Mell  in- 
Fourier  correlator  is  evaluated  with  respect  to  small  scale  differences  in  the 
input.  The  effect  of  different  amplitude  distributions  of  added  noise  is  noted 
on  the  peak  value  and  shape  of  the  correlation  image. 

The  Mel  1  in-Fourier  image  correlation  process  is  two-dimensional  in  nature, 
but  insight  into  the  ln(R),9  mapping  operation  and  the  correlation  energy 
distribution  is  gained  when  results  are  presented  in  the  form  of  one-dimen¬ 
sional  plots.  In  this  research,  photographs  taken  from  the  television  monitor 
of  an  image  processing  computer  are  used  to  portray  the  nature  of  the  entire 
Mel  1  in-Fourier  process,  but  one-dimensional  plots  are  used  to  show  details 
of  the  energy  distribution  in  the  correlation  Image.  Images  are  characterized 
by  a  plot  of  the  intensity  values  along  a  line  passing  through  the  peak  value 
of  the  image. 

Additional  experimental  techniques,  such  as  range  adaptive  bandpass,  are 
applied  to  extend  the  normal  scale  invariance  of  the  Mel  1  in-Fourier  type  corre¬ 
lator.  Anderson  and  Call  ary,  Reference  4,  calculated  only  the  value  of  the 
correlation  peak  as  a  figure  of  merit  for  the  Mel  1  in-Fourier  correlator. 

This  investigation  calculates  the  entire  correlation  image  plane.  Using  this 
technique,  the  advantages  of  spatial  filtering  are  made  clear. 
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In  the  logarithmic  coordinate  transformation.  In  their  scheme,  oversampling  at 
small  values  of  radial  distance  was  governed  by  meeting  the  Nyqulst  criterion 
at  the  widest  sampling  Interval,  and  by  oversampling  only  to  the  extent  neces¬ 
sary  to  accommodate  the  predicted  scale  change.  Using  this  scheme,  space 
bandwidth  requirements  Increased  by  a  factor  of  only  2.7  when  accommodating  a 
100  percent  scale  change. 

Variations  of  this  geometrical  transformation  scheme  have  been  reported 
by  other  researchers.  Reference  5.  In  those  studies,  the  location  of  the 
correlation  peak  In  the  Mell in-Fourier  correlation  plane  was  used  to  Identify 
the  actual  scale  change  and  rotation  of  the  sensed  Image.  Then,  a  new  refer¬ 
ence  Image  was  synthesized  at  the  proper  scale  and  rotation  angle  for  applica¬ 
tion  of  traditional  correlation  schemes. 


4.  AREAS  INVESTIGATED  IN  THIS  WORK 

This  study  evaluates  the  success  of  a  pattern  recognition  system  which 
attempts  to  recognize  objects  In  a  sensed  image  which  differ  in  scale  and 
rotation  from  the  reference  Image.  The  intent  is  to  establish  the  limits  of 
use  for  the  Mel  1  In-Fourier  correlation  techniques  rather  than  to  design  a 
specific  algorithm  for  pattern  recognition.  Experience  has  shown  that  such 
algorithms  are  very  dependent  upon  a  system  application. 

The  approach  used  here  was  to  calculate  sampling  requirements  based  on  a 
desired  range  of  scale  Invariance,  and  then  to  use  a  computer  simulation  of  the 
correlation  process  to  calculate  system  performance.  The  scale  of  the  sensed 
Image  was  then  allowed  to  exceed  the  design  limits  of  the  system  to  find  the 
effect  on  the  correlation  peak.  The  correlation  coefficient  and  the  shape 
of  the  correlation  peak  were  used  as  figures  of  merit  to  judge  the  success  of 
each  trial. 

This  study  applies  the  theory  of  Mell in-Fourier  correlation  to  imagery 
collected  by  a  sensor  carried  aboard  a  helicopter.  The  correlator  modeled  is 
Intended  to  represent  an  optical  implementation  of  a  scale  invariant  system. 
Such  an  optical  signal  processing  system  can  be  modeled  realistically  by  numer¬ 
ical  experiments.  The  limited  dynamic  range  of  optical  systems  Is  simulated  by 


RADIAL  SPATIAL  FREQUENCY  (CYCLES/FRAME) 


Figure  6.  Amplitude  along  a  line  passing  through 
the  center  of  the  R,9  mapped  Image  if  a  10  X  10 
square.  Normalized  from  0  to  255. 
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RADIAL  SPATIAL  FREQUENCY  (CYCLES/FRAME) 


Figure  7.  Amplitude  along  a  line  passing  through 
the  center  of  the  R,9  mapped  Image  of  a  20  X  20 
square.  Normalized  from  0  to  255. 


NORMALIZED  LOG  FREQUENCY  (CYCLES/FRAME) 

Figure  8.  Amplitude  along  a  line  passing  through 
the  center  of  ln(R),9  mapped  image  of  a  10  X  10 
square.  Normalized  from  0  to  255. 


ZED  AMPLITUDE 


LOGARITHMIC  POLAR  TRANSFORM 

SENSED  20  X  20  SQUARE 


NORMALIZED  LOG  FREQUENCY  (CYCLES/FRAME) 

Figure  9.  Amplitude  along  a  line  passing  through 
the  center  of  the  ln(R),9  mapped  Image  of  a  20  X  20 
square.  Normalized  from  0  to  255. 


degrade  as  the  annulus  chosen  for  the  transformation  becomes  smaller  and 
smaller.  This  occurs  at  very  close  ranges,  and  Is  not  critical  If  the 
reference  Image  Is  made  at  the  closest  expected  range.  In  a  system  with  the 
ability  for  range  adaptive  bandpass,  range  Information  Is  used  to  change  the 
transformation  parameters  to  track  the  critical  region  of  spatial 
frequencies  In  the  Fourier  transform  plane.  The  cross  correlation  peak 
should  remain  high  over  a  wider  range  of  scale  change  than  when  the  peak  Is 
reduced  due  to  scale  changes  causing  Irrelevant  Information  to  enter  the 
transformed  space. 

Figures  10  through  13  show  the  results  when  the  principles  discussed 
above  are  applied  to  real  Images.  The  sequence  of  nine  Images  that  make  up 
each  figure  show  the  Mel  1  In-Fourier  correlation  process  outlined 
schematically  In  Figure  1.  The  upper  left  pair  of  Images  labeled  "REFERENCE" 
and  "SENSED"  are  Infrared  Images  of  an  armored  vehicle.  The  right  image  of 
the  pair  has  been  rotated  about  Its  center  by  45  degrees.  The  second  pair 
of  images  labeled  "REFERENCE  FFT"  and  "SENSED  FFT"  are  the  magnitudes  of  the 
Fourier  transforms.  The  Image  pair  labeled  "LOG  R,  THETA  TRANSFORMS"  are 
the  ln(R),0  ‘  mappings  done  from  the  Fourier  transform  images  Immediately 
above.  The  right  ln(R),0  Image  shows  that  rotation  In  the  sensed  Image 
produces  vertical  translation  of  the  pattern  In  the  ln(R),0  Image.  The  images 
labeled  "MELLIN  TRANSFORMS"  are  the  magnitudes  of  the  Fourier  transforms  of 
the  Images  Immediately  above.  Since  the  only  difference  between  the  refer¬ 
ence  and  sensed  1n(R),0  Images  Is  a  vertical  shift,  the  magnitude  Images 
labeled  "MELLIN  TRANSFORMS"  are  nearly  Identical.  The  last  step  In  the 
Mell In-Fourier  correlation  process  Is  shown  as  the  rightmost  Image  labeled 
"CROSS  CORRELATION."  The  real  and  Imaginary  components  of  the  ln(R),0 
Images  were  used  to  compute  the  cross  correlation  Image  plane.  The  dark 
spot  In  the  correlation  Image  Is  enhancement  performed  by  the  Image  process¬ 
ing  computer  to  make  the  peak  In  the  correlation  plane  more  visible.  The 
vertical  position  of  the  correlation  peak  reveals  the  correct  rotation 
difference  between  the  reference  and  sensed  Images,  l.e.,  45  degrees,  and 
the  amplitude.  Indicated  by  the  peak  Intensity,  shows  the  degree  of 
similarity  between  the  reference  and  sensed  images. 

Figure  10  shows  that  rotation  of  the  input  image  by  45  degrees  may  be 
detected  using  standard  space  variant  processing  techniques.  In  this 


EFFECT  QF  ROTATiO 


Figure  10.  Sequence  of  Images  Showing  the  Mel  1  in-Fourier  Correlation 

Process.  The  sensed  image  has  no  scale  change,  but  is  rotated  by  45 
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Figure  11.  Sensed  Image  Scaled  by  a  Factor  A  =  2.0  with  no  Rotation 


Figure  12.  Sensed  Image  Scaled  by  a  Factor  A  =  2.o  with  45°  Rotation 


example,  no  scale  change  was  Involved,  and  pattern  recognition  was  achieved 
with  a  normalized  correlation  coefficient  of  .9.  In  Figure  11,  the  sensed 
Image  was  scaled  by  a  factor  of  2.0,  and  no  rotation  was  present.  The 
Images  labeled  "LOG  (R),  THETA  TRANSFORMS*  show  what  happens  when  a  scale 
change  causes  Information  to  leave  the  In  (R),0  plane.  The  correlation 
coefficient  Is  reduced  to  .7,  even  though  Its  position  is  displaced  from  the 
center  of  the  correlation  plane  by  the  correct  amount,  indicating  a  scale 
change.  Figure  12  shows  that  combining  a  scale  change  with  a  rotation 
reduces  the  correlation  coefficient  still  further  to  .6,  with  the  appearance 
of  noise  at  high  spatial  frequencies  being  evident  in  the  In  (R),0  images. 
Application  of  range  adaptive  bandpass  in  Figure  13  restores  the  correlation 
coefficient  to  .89,  with  the  rotation  being  correctly  sensed.  However  the 
cability  to  detect  the  amount  of  scale  change  by  the  position  of  the  correla¬ 
tion  peak  has  been  given  up  by  adapting  the  In  (R),0  transform  equations 
using  range  information  from  an  independent  source. 

2.  TESTING  THEORETICAL  PREDICTIONS 

a.  Predicted  Scaling  Limits 

The  success  of  the  application  of  Mel  1  in-Fourier  correlation  to  the 
tactical  images  shown  in  Figures  10  through  13  led  to  a  careful  analysis  of 
the  scale  invariance  that  should  be  expected  of  the  Mel  1  in-Fourier  correlator 
when  used  with  a  particular  image  frame  size.  The  images  used  in  predicting 
the  expected  scale  invariance  were  centered  in  a  frame  of  256  by  256  and 
were  scaled  by  sampling  in  a  polar  coordinate  system  in  which  R  was  multi¬ 
plied  by  a  scaling  constant  A.  The  value  of  A  was  used  as  an  index  to 
denote  the  amount  of  scaling  present  in  each  trial.  An  image  of  the  Fourier 
transform  of  the  reference  scene,  a  side  view  of  an  armored  vehicle,  was 
inspected  to  identify  the  most  critical  features  for  pattern  recognition.  The 
selection  of  "most  critical"  features  in  the  Fourier  transform  plane  was 
subjective  to  the  extent  that  some  spatial  frequencies  were  identified  with 
the  reference  image  window  and  were  eliminated  by  specifying  the  lower  limit 
of  the  filter.  Also,  knowledge  of  imager  characteristics  as  well  as  inspec¬ 
tion  of  large  numbers  of  Fourier  transform  images  allowed  an  upper  limit  to 
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be  specified  for  the  spatial  bandpass.  Limiting  the  frequencies  used  for 
transformation  Into  the  ln(R),8  plane  allowed  the  frequencies  to  be 
adjusted  for  the  scale  change  present  in  the  sensed  image.  The  adaptation 
of  the  filter  according  to  scale  change  Is  a  capability  that  is  often 
present  in  tactical  pattern  recognition  systems  with  radar  altimeters  or 
laser  rangers. 

Calculations  were  done  using  the  side  view  of  the  armored  vehicle  as  a 
reference  image  to  verify  that  the  distinct  features  in  the  Fourier  trans¬ 
form  plane  were  due  to  the  spacing  and  number  of  the  vehicle  road  wheels. 

The  window  or  frame  size  of  the  reference  image  was  used  to  calculate  the 
location  of  the  first  zero  crossing  in  the  Fourier  transform  plane.  These 
calculations  were  patterned  after  the  example  given  in  Section  II,  under 
Sample  Adaptive  Calculation."  The  filter  limits  calculated  using  the 
methods  described  above  resulted  in  an  annulus  in  the  Fourier  transform 
plane  between  20  and  110  pixels.  For  range  adaptation,  the  lower  and  upper 
limits  of  this  annulus  were  scaled  according  to  the  factor  A  used  for  the 
sensed  image.  The  inverse  relationship  between  scaling  in  the  image  plane 
and  scaling  in  the  Fourier  transform  plane  caused  the  filter  boundaries  to 
be  multiplied  by  1/A  whenever  the  sensed  image  was  scaled  by  A.  The  conven¬ 
tion  used  in  this  research  was  that  values  of  A  greater  than  1.0  resulted  in 
a  smaller  image  but  a  larger  Fourier  transform.  Values  of  A  less  than  1.0 
denoted  a  larger  Image  and  a  smaller  Fourier  transform. 

The  scaling  limits  predicted  for  the  example  discussed  above  were  deter¬ 
mined  from  Nyquist  sampling  and  spectral  resolution  requirements.  It  was 
observed  from  Inspection  of  the  Fourier  transform  of  the  reference  image 
that  this  imagery  was  not  corrupted  by  aliasing  effects,  and  that  the 
lower  frequency  limit  of  20  pixels  still  provided  an  adequate  number  of 
samples  to  define  a  spectral  lobe  when  scaling  by  a  factor  of  A=.3  was 
performed.  When  scaling  by  factors  larger  than  A-1.5,  it  was  observed  that 
loss  of  high  frequency  information  pertaining  to  road  wheel  spacing,  e.g., 
the  second  lobe  moving  out  of  the  sampled  domain,  would  violate  the  Nyquist 
condition.  Therefore  the  predicted  range  of  scale  invariance  for  the  corre¬ 
lator  chosen  for  this  example  was  from  A=.3  to  A=1.5. 

Figure  14  shows  the  results  obtained  using  the  computer  simulation  of 
the  Mel  1  in-Fourier  correlator.  The  points  labeled  with  a  square  symbol  were 
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EXTENDING  SCALE  INVARIANCE 


Figure  14,  Scale  Invariance  extended  by  adapting 
the  spatial  filter. 


obtained  from  a  correlator  with  the  spatial  filter  fixed  at  20  to  110 
pixels.  The  filter  did  not  adapt  to  scale  changes  in  the  sensed  image.  The 
correlation  coefficient  steadily  decreases  from  .9  with  no  scale  change,  and 
drops  below  the  threshold  of  .7  if  A  is  less  than  .5.  The  points  labeled 
with  cross  symbols  were  obtained  from  a  correlator  in  which  the  spatial  filter 
adapted  to  the  scale  change  present  in  the  sensed  image.  The  annulus 
defined  in  the  Fourier  transform  plane  for  the  In  (R),0  transformation  was 
allowed  to  change  size  according  to  the  scaling  factor  A.  This  adaptation 
of  the  spatial  filter  allowed  the  correlation  coefficient  to  remain  fairly 
constant  over  the  entire  range  of  predicted  scale  change.  The  correlation 
coefficient  for  the  adaptive  correlator  does  not  drop  below  .7  until  A 
becomes  smaller  than  .1.  This  represents  an  extension  of  the  scale  invari¬ 
ance  for  small  values  of  A  by  a  factor  of  5.  The  performance  of  the  corre¬ 
lator  when  A  was  greater  than  1.0  was  similar  for  the  fixed  and  for  the 
adaptive  filter.  At  A*1.2  the  upper  frequency  limit  of  the  adaptive  filter 
moved  to  a  value  of  128,  which  was  the  edge  of  the  available  bandwidth. 

Since  the  filter  could  no  longer  adapt,  the  correlators  behaved  similarly. 
These  results  show  that  adaptive  filtering  may  be  used  to  extend  the  normal 
scale  invariance  of  the  Mel  1  in-Fourier  correlator  to  allow  pattern  matches 
with  high  confidence  (high  correlation  coefficient)  even  when  operating  near 
the  lower  limits  for  the  scaling  factor  A. 
b.  Failure  Modes 

Data  acquired  while  investigating  the  scaling  limits  of  the  Mell  in- 
Fourier  correlator  caused  interest  in  information  that  could  be  obtained 
from  the  correlation  plane  even  after  the  correlator  had  "failed"  by  giving 
a  correlation  coefficient  less  than  .7.  The  position  of  the  peak  value  in 
the  correlation  plane  reveals  the  amount  of  scale  change  and  rotation  differ¬ 
ence  sensed,  and  this  information  seems  to  be  reliable  even  when  the  value 
of  the  correlation  peak  is  less  than  that  required  for  a  confident  pattern 
match.  Figure  15  is  a  plot  of  the  scale  change  detected  by  the  X  position  of 
the  correlation  peak  versus  the  actual  s-cale  change  present  in  the  sensed 
image.  As  before,  the  square  symbols  show  correlation  done  with  a  fixed 
filter,  and  the  cross  symbols  show  correlation  done  with  an  adaptive  filter. 
The  fixed  filter  gives  incorrect  values  of  scale  change  when  the  scaling 
factor  A  goes  below  .5,  as  predicted  by  sampling  theory.  The  adaptive 
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Figure  15.  Detected  scale  change  extended  by  adapting 
the  spatial  filter. 
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The  techniques  developed  for  Mel  1  in-Fourier  correlation  were  tested  in 
an  investigation  of  the  benefits  of  higher  space  bandwidth.  The  effect  of 
the  larger  format  256  by  256  frame  was  to  increase  the  system  space  band¬ 
width  by  a  factor  of  four.  When  standard  scale  invariant  techniques  were 
applied  to  the  large  format  frame  in  the  same  way  as  with  the  128  by  128 
image,  the  larger  space  bandwidth  produced  a  modest  improvement  in  the 
correlation  peak;  not  a  significant  improvement  considering  the  large  price 
paid  in  increasing  the  system  space  bandwidth.  When  range  adaptive 
techniques  were  applied  to  the  Mell In-Fourier  correlator  using  the  larger 
frame  size,  the  correlation  peak  value  still  did  not  show  a  significant 
increase.  These  experiments  suggest  that  signal  processing  techniques  such 
as  range  adaptive  transformations  and  spatial  filtering  contribute  more  to 
the  success  of  the  Mel  1  in-Fourier  correlator  than  an  increase  in  correlator 
space  bandwidth.  The  experiments  further  suggest  that  the  theory  for 
predicting  performance  of  Mel  1  in-Fourier  correlator  may  be  based  on  the 
well-established  principles  of  Nyquist  sampling  and  spectral  resolution. 

Investigations  into  the  sensitivity  of  the  Mel  1  in-Fourier  correlator  to 
noise  measured  the  influence  of  both  uniform  and  Gaussian  noise  distribu¬ 
tions  on  the  value  and  position  of  the  correlation  peak.  Higher  space 
bandwidths  were  investigated  and  sensitivity  to  added  noise  was  evaluated 
using  the  space  bandwidth  available  in  a  256  by  256  frame.  Optimal  choice 
of  the  spatial  filter  used  in  processing  the  sensed  image  made  the  Mell in- 
Fourier  correlator  less  sensitive  to  Gaussian  noise.  Experiments  with 
incorrectly  matched  library  Images  verified  that  the  correlator  was  not 
simply  responding  to  the  scaled  window  through  which  the  sensed  image  was 
viewed. 


SECTION  V 
CONCLUSIONS 

The  numerical  experiments  described  In  this  paper  have  shown  how  several 
Issues  can  affect  the  evaluation  of  the  space  bandwidth  requirements  of  a 
scale  Invariant  correlation  system.  Spectral  resolution  and  Nyqulst 
sampling  were  seen  to  be  the  two  main  determinants  of  the  space  bandwidth 
required  to  achieve  correlator  Invariance  over  a  given  range  of  scale 
change.  Other  things,  such  as  windowing  and  truncation  of  the  Fourier 
transform  plane,  were  shown  to  have  an  effect  on  the  validity  of  the  cross 
correlation  image. 

Calculations  done  under  the  assumptions  used  by  Anderson  and  Callary, 
Reference  4,  were  used  to  see  how  savings  in  system  space  bandwidth  could  be 
realized  by  careful  choice  of  geometric  transformation  parameters.  Further 
refinements  of  the  choice  of  transformation  parameters  were  applied  in  the 
development  of  the  technique  of  range  adaptive  bandpass. 

The  series  of  examples  presented  in  the  section  discussing  optimum 
sampling  for  scale  invariance  showed  how  each  of  the  principles  described 
In  the  preceding  sections  could  be  used  successfully  to  restore  the  ampli¬ 
tude  of  the  cross  correlation  peak  in  a  scale  invariant  correlator. 

The  theoretical  scaling  limits  of  a  Mell in-Fourier  correlator  were 
calculated  and  a  computer  simulation  was  used  to  verify  that  the  theoretical 
benefits  of  range  adaptive  filtering  could  be  applied  in  practice.  Using  only 
the  techniques  of  range  adaptive  sampling  and  careful  choice  of  transforma¬ 
tion  parameters,  the  scale  invariance  of  the  Mell in-Fourier  correlator  was 
extended  by  a  factor  of  5  for  scaling  factors  less  than  1.0  without  increas¬ 
ing  the  space  bandwidth  of  the  system.  The  performance  of  the  Mell  in- 
Fourier  correlator  which  used  a  range  adaptive  filter  was  similar  to  the 
fixed  filter  correlator  when  the  scaling  factor  A  was  greater  than  1.0. 

Design  approaches  were  suggested  for  systems  required  only  to  detect 
scale  and  rotation  changes  in  a  sensed  image  rather  than  to  perform  a 
detailed  measure  of  pattern  match.  Range  adaptive  filtering  techniques  were 
seen  to  provide  accurate  measures  of  scale  and  rotation  changes  even  though 
scale  changes  in  the  sensed  image  had  violated  Nyquist  sampling  conditions. 
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Figure  19.  Gaussian  distributed  noise  added  to  the 
sensed  Image.  Added  noise  was  zero  mean,  with  the 
three  standard  deviations  shown.  Spatial  filter 
optimized  to  the  scale  change  In  the  sensed  Image. 


change  present  In  the  sensed  Image.  When  the  sensed  spatial  filter  was 
optimized  for  the  scale  change,  Gaussian  noise  was  again  Introduced  Into  the 
sensed  Image  to  determine  the  effect  on  the  correlation  curve.  Figure  19 
shows  that  use  of  a  filter  optimized  for  the  scale  change  In  the  sensed 
Image  can  restore  the  correlation  peak  to  values  required  for  full  confi¬ 
dence  In  a  pattern  match.  The  shape  of  the  correlation  curve  In  Figure  19 
Is  significant  because  the  secondary  peak,  present  In  Figure  18  when  the 
filter  was  not  optimized.  Is  reduced.  The  secondary  peak  In  Figure  18 
represents  a  partial  match  caused  by  energy  present  In  the  window  of  the  in 
(R) ,0  image.  The  location  of  the  secondary  peak  is  at  the  center  of  the 
correlation  plane,  as  expected,  and  represents  a  match  between  the  reference 
and  sensed  windows.  Figure  19  also  shows  that  the  level  of  Gaussian  noise 
necessary  to  reduce  the  correlation  peak  below  .7  is  higher  when  the  sensed 
filter  is  optimized  for  scale  change. 

d.  Demonstration  of  Insensitivity  to  Window 

The  results  shown  in  Figure  19  caused  concern  that  most  of  the 
amplitude  observed  In  the  correlation  peaks  of  Figures  16  through  19  was 
caused  by  the  window,  or  frame  size,  of  the  sensed  Image.  The  scaling 
algorithm  used  to  produce  precisely  scaled  sensed  imagery  also  served  to 
scale  precisely  the  Image  window  by  the  same  factor.  Since  the  position  of 
the  secondary  peak  did  not  change  with  the  scaling  factor  of  the  sensed 
Image,  It  was  considered  likely  that  correlation  amplitude  was  generated  by 
the  In  (R),8  window  size,  which  was  Independent  of  the  object  scaling 
factor.  An  experiment  was  performed  to  demonstrate  that  the  correlation 
peak  was  Indicating  the  correct  scale  change  and  degree  of  similarity 
between  the  reference  image  and  the  sensed  Image.  The  Mel  1  In-Fourier  system 
was  made  to  perform  a  correlation  between  the  usual  reference  image  and  an 
Incorrect  sensed  image.  The  incorrect  sensed  Image  was  scaled  by  a  factor 
of  A  ■  .7  and  the  filter  bandpass  was  optimized  for  this  scale  change.  The 
resulting  correlation  peak  was  not  expected  to  indicate  the  proper  scale  factor 
nor  was  It  expected  to  have  sufficient  amplitude  to  indicate  a  pattern 
match.  The  amplitude  of  the  correlation  peak  was  0.662,  well  below  the 
threshold  for  match,  and  the  X  position  of  the  correlation  peak  indicated  a 
scale  factor  of  .85.  These  results  indicate  that  the  amplitude  and  position 
of  the  correlation  peak  are  reliable  Indicators  of  pattern  match  and  scale 
factor,  and  are  not  completely  determined  by  image  window  size. 
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Figure  18.  Gaussian  distributed  noise  added  to 
sensed  Image.  Added  noise  was  zero  mean,  with  the 
three  standard  deviations  shown. 
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with  the  evaluations  Involving  scale  change,  the  correlation  Image  was 
represented  by  a  horizontal  scan  through  the  correlation  peak.  The  three 
curves  shown  represent  altered  histogram  widths  of  25,  50,  and  128  counts; 
each  region  centered  around  the  Image  mean  value  of  59  counts.  Figure  17 
shows  that  Introduction  of  noise  causes  the  cross  correlation  curve  to 
decrease  smoothly  past  the  threshold  amplitude  considered  necessary  for 
pattern  match. 

b.  Effect  of  Adding  Gaussian  Noise 

A  more  physically  significant  Gaussian  distribution  was  used  in  a 
second  series  of  trials  with  the  same  sensed  Image.  The  Gaussian  distribu¬ 
tion  was  achieved  by  making  use  of  the  Central  Limit  theorem  applied  to  the 
uniformly  distributed  random  function  described  above.  The  mathematical 
form  for  the  transformation  was 
K  *  48 

Y  ■  T  [  (RAN  (SEED)  -  .5)  /  2  ] 

K  «  1 

A  new  random  variable,  Y',  with  the  desired  mean  of  U  and  stan¬ 
dard  deviation  V  was  obtained  using  the  transformation 

Y'  »  U  +  V  *  Y 

The  results  of  trials  conducted  with  Gaussian  distributed  random  noise  are 
shown  In  Figure  18.  The  three  curves  represent  the  shape  of  the  correlation 
curve  after  the  addition  of  noise  having  standard  deviations  of  0,  10.0,  and 
20.0.  The  horizontal  scale  of  Figure  18  was  changed  to  see  better  that  the 
correlation  curve  still  has  a  discernible  peak  which  is  located  at  the 
proper  scale  and  rotation  coordinates  even  though  the  amplitude  of  the 
correlation  peak  has  fallen  below  the  threshold  vaalue  of  .7.  This  charac¬ 
teristic  of  the  correlation  Image  suggests  that  the  ability  of  the  Mel  1  in- 
Fourier  correlator  to  detect  scale  and  rotation  changes  may  be  preserved 
even  when  noise  levels  have  reduced  the  confidence  in  pattern  matches  between 
reference  and  sensed  images. 

c.  effect  of  Optimal  Spatial  Filter 

Ouring  the  course  of  the  Investigation  Into  noise  sensitivity,  the 
spatial  filter  used  to  eliminate  low  frequencies  in  the  Fourier  transform 
plane  was  fixed  for  the  reference  and  sensed  Images  to  pass  all  frequencies 
between  radii  of  10  and  128  pixels.  The  filter  was  not  adapted  to  the  scale 
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Figure  17.  Uniformly  distributed  noise  added  to 
v  sensed  Image.  Added  noise  was  centered  around  the 

Image  mean  value,  and  had  the  three  histogram  widths 
I  shown. 
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CORRELATION  COEFFICIENT 


MELLIN  — FOURIER  CORRELATION 


EFTECT  or  SCALE  CHANGE 


COLUMN  OF  CORRELATION  IMAGE 
O  A  -  1.0  +  A  m  2.0  A  A  =»  0.5 


Figure  16.  Shape  of  the  correlation  curve  for  two 
extreme  values  of  scale  change.  A*. 5  and  A=2.0  . 

Range  adaptive  spatial  filter  not  applied.  The  curve 
for  A«1.0  represents  a  sensed  image  with  no  scale 
change. 
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2.  A00IN6  NOISE 


a.  Effects  of  Adding  Uniform  Noise 

A  further  Investigation  of  the  Mel  1  In-Fourier  correlation  scheme  was 
performed  by  adding  noise  to  the  sensed  Image  and  noting  the  effect  on  the 
shape  and  magnitude  of  the  correlation  peak.  The  Intent  of  this  part  of  the 
Investigation  was  to  simulate  the  Imagery  collected  by  a  sensor  that  might 
be  corrupted  by  various  noise  sources.  The  Image  used  as  a  reference  was 
the  same  as  the  reference  Image  In  tests  involving  changes  In  scale  and 
rotation,  i.e.,  one  which  contained  no  added  noise.  A  random  variable 
function  from  a  computer  statistical  library  was  used  which  was  capable  of 
generating  a  pseudo  random  number  uniformly  distributed  between  0  and  1.0. 
The  function  Is  considered  pseudo  random  because  the  seed,  or  initializing 
Integer,  completely  determines  the  string  of  random  digits  generated  by  the 
function  each  time  it  Is  used.  However,  the  probability  distribution  of  the 
digits  produced  by  the  function  is  the  desired  uniform  distribution.  For 
initial  investigations  into  the  sensitivity  of  the  Mel  1  in-Fourier  correlator 
to  added  noise,  the  sensed  image  was  treated  as  a  random  variable  to  which 
another  random  variable  with  the  desired  statistical  distribution  was  added. 
The  addition  of  noise  took  the  mathematical  form 
NOISY  SENSED  IMAGE  =  SENSED  IMAGE  +  255  X  NOISE  X  RAN  (SEED) 
where 

NOISE  =  value  from  0  to  1.0  providing  a 

measure  of  the  amount  of  added  noise 
RAN  (SEED)  *  random  variable  uniformly 
distributed  from  0  to  1.0 

The  mathematical  form  above  produced  aa  sensed  image  with  an  altered  ampli¬ 
tude  distribution  between  values  determined  by  the  noise  measuring  variable 
NOISE.  The  total  amplitude  distribution  of  the  sensed  image  was  the  combi¬ 
nation  of  uniformly  distributed  noise  and  the  original  distribution.  The 
effect  of  varying  the  value  of  NOISE  was  to  extend  the  area  of  the  image 
histogram  over  which  the  added  noise  had  its  effect.  One  meaningful  measure 
of  this  type  of  noise  is  the  width  of  the  histogram  region  that  is  altered. 
The  degradation  of  the  correlation  peak  when  the  sensed  image  was  trans¬ 
formed  by  noise  of  uniform  amplitude  distribution  is  shown  in  Figure  17.  As 
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scaled  or  zoomed  around  the  center  of  the  reference  Image  as  well  as  rotated 
by  the  desired  angle.  When  higher  space  bandwidth  was  used  In  the  experi¬ 
ment  shown  In  Figure  12,  with  scaling  and  rotation  present,  the  correlation 
peak  value  changed  from  .6  to  .7.  This  Increase  was  not  considered  signifi¬ 
cant  since  the  correlation  value  was  so  close  to  the  threshold  of  .7  consid¬ 
ered  necessary  for  pattern  match.  The  threshold  value  of  .7  for  pattern  match 
was  experimentally  determined  by  correlation  experiments  between  both 
geometric  shapes  and  real  objects.  In  these  experiments,  even  dissimilar 
objects  produced  correlation  coefficients  of  .6.  When  range  adaptation  of  the 
transformation  equations  was  performed  as  in  Figure  13  but  using  the  higher 
space  bandwidth,  the  correlation  coefficient  changed  from  0.89  (with  the  128 
by  128  frame)  to  0.92.  This  slight  increase  was  not  considered  significant, 
considering  the  price  paid  In  a  four-fold  increase  in  the  number  of  sample 
points  required. 

b.  Effects  of  Exceeding  the  Correlator  Design 

A  detailed  study  of  the  effect  of  scale  change  on  the  correlation 
peak  value  and  curve  shape  was  performed  on  the  images  with  the  higher  space 
bandwidth.  The  rotation  angle  between  the  reference  image  and  the  sensed 
Image  was  fixed  at  45  degrees  to  be  sure  that  the  location  of  the  correla¬ 
tion  peak  still  revealed  the  correct  rotation  angle.  Scale  factors  of  the 
sensed  image  were  varied  between  .5  and  2.0,  and  Figure  16  shows  a  plot  of 
the  values  of  the  correlation  images  along  a  line  passing  through  the  peak 
value.  The  two  curves  with  the  legends  "A  =  .5"  and  "A  =  2.0"  represent  the 
extreme  values  over  which  the  Mel  1  in-Fourier  correlation  scheme  was 
exercised.  The  shape  of  the  correlation  curve  for  no  scale  change  is  shown 
In  the  curve  labeled  "A  *  1.0."  Symbols  are  Included  In  the  plots  to  define 
the  curves  when  they  cross  each  other.  The  Mel  1  in-Fourier  correlator  in 
this  example  was  designed  to  cover  a  range  of  scale  change  from  A  =  2.0 
to  a  A  *  .5  without  the  use  of  range  adaptive  techniques.  Figure  16  shows 
that  the  value  of  the  correlation  peak  at  these  extreme  scale  changes  drops 
just  below  the  threshold  value  of  .7.  The  range  of  scale  change  over  which 
the  system  could  operate  In  this  example  was  not  extended  using  range  adap¬ 
tive  techniques.  The  use  of  higher  space  bandwidth  has  increased  the  range 
of  scale  Invariance  of  the  correlator  only  by  the  it  that  was  predicted 
from  sampl ing  theory. 


SECTION  IV 

ADDITIONAL  EXPERIMENTS 


1.  HIGHER  SPACE  BANOWIOTH 

The  tests  shown  in  Figures  10  through  13  were  repeated  with  Images 
having  a  space  bandwidth  of  256  by  256  pixels.  The  larger  Image  format  was 
equivalent  to  sampling  the  reference  and  sensed  patterns  at  a  higher 
frequency.  To  create  the  Fourier  transform  patterns,  the  128  by  128  input 
images  were  simply  centered  In  the  256  by  256  frames.  No  additional  informa¬ 
tion  was  added  to  the  Input  Image,  but  the  Fourier  transform  calculated  over 
the  256  by  256  frame  extended  to  higher  spatial  frequencies.  This  added  space 
bandwidth  allowed  for  more  shift,  i.e.,  more  scale  change,  in  the  In  (R) ,0 
Images  when  scale  changes  caused  distortions  In  the  Fourier  transform  plane. 
The  annulus  of  spatial  frequencies  chosen  for  logarithmic  resampling  was 
chosen  using  the  same  criteria  suggested  In  discussions  of  the  range  adap¬ 
tive  technique;  i.e.,  frequency  boundaries  were  established  using  approxi¬ 
mate  ranges  supplied  with  the  test  Imagery. 

a.  Expected  Benefits  of  Higher  Space  Bandwidth 

The  calculations  of  the  upper  and  lower  spatial  frequencies  critical 
for  pattern  recognition  may  be  done  without  knowledge  of  the  system  space 
bandwidth  available.  With  the  larger  format  frames  however,  the  spatial 
frequency  boundaries  could  extend  to  larger  distances,  I.e.,  more  pixels, 
from  the  center  or  d.c.  term.  With  the  larger  space  bandwidth  available  in 
a  256  by  256  format.  It  was  expected  that  It  would  be  possible  to  vary  scale 
over  a  wider  range  while  maintaining  the  value  of  the  correlation  coeffi¬ 
cient.  Scale  change  was  performed  using  a  sample  and  hold  technique,  with 
the  sensed  image  remaining  centered  In  the  frame  for  all  scale  changes.  The 
sample  and  hold  technique  was  Implemented  using  the  same  equations  to  gene¬ 
rate  sample  points  that  were  used  to  rotate  the  reference  Image.  The  sample 
coordinates  for  the  sensed  Image  had  to  be  generated  in  polar  form  in  order 
to  add  the  required  rotation  angle.  To  generate  a  scaled  Image,  the  radial 
coordinate  for  the  sample  points  was  simply  multiplied  by  a  scaling  factor  A. 
The  effect  of  this  sampling  scheme  was  to  provide  a  sensed  image  that  was 
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filter  gives  correct  values  of  scale  change  (within  20  percent)  when  A*.05. 

The  rotation  reported  by  both  correlators  was  within  10  percent  of  the  correct 
value  for  the  entire  range  of  scale  changes  tested.  With  larger  values  of 
A,  both  correlators  became  Insensitive  to  scale  change,  probably  because  the 
spectral  features  become  larger  and  occupy  more  of  the  ln(R)t8  plane, 
making  small  shifts  In  cross  correlation  harder  to  detect.  This  result 
suggests  that  reliable  Indications  of  scale  and  rotation  may  be  obtained 
from  a  Mell In-Fourier  correlator  using  an  adaptive  filter  even  when  the 
designed  scale  change  has  been  exceeded.  Correlator  system  design  may  be 
extended  to  cover  values  of  scale  change  not  allowed  by  sampling  theory  If 
information  about  scale  change  and  rotation  are  required  rather  than  the 
degree  of  pattern  match. 
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